Automatic design-creating artificial neural network device and method, using ux-bits

ABSTRACT

The present disclosure relates to a development project collaboration method using a UX-bit of a program such as an application used by customers and service providers. Specifically, the present disclosure relates to a development project collaboration method capable of reducing a development period and costs using a UX-bit such that elements constituting a project are easily searched for and applied. The development project collaboration method includes receiving a project goal, receiving detailed tasks for achieving the project goal, arranging content agreed upon in the project goal and the detailed tasks into UX-bits in an integrated function definition, combining the UX-bits based on the function definition and arranging information architecture (IA) elements, constituting a wireframe based on the IA elements, designing a configuration page based on the wireframe, providing a verification page for verifying the configuration page formed by combining the UX-bits, and when a modification occurs in at least one of the project goal, the detailed task, the UX-bit, the IA element, the wireframe, and the design, simultaneously modifying the project goal, the detailed task, the UX-bit, the IA element, the wireframe, and the design to which the modification is reflected in the entire project.

TECHNICAL FIELD

The present disclosure relates to an automatic design generatingartificial neural network device and method using a user experience(UX)-bit. Specifically, the present disclosure relates to a developmentproject collaboration method in which, designs are automaticallygenerated in response to input of elements constituting a graphic designproject, e.g., a web, an application (app), or the like, therebyenabling reduction of development period and costs.

BACKGROUND ART

With the development of IT technology, the development of platformsapplying information and communication technology and the introductionof e-commerce are becoming more active, and the number of companies, ofwhich a business form is converted from offline to online or which addonline channels in addition to existing offline businesses, is beingincreased. In addition to the continuous expansion of the onlinebusiness market, the demand for non-face-to-face related services israpidly increasing due to the recent outbreak of coronavirus.

The development demand for software and user interface (UI)/userexperience (UX) necessary for telecommuting and remote services isincreasing, and the life cycle of UI/UX designs is rapidly changing dueto the rapid flow of the times.

There is a need for UI/UX development tools that can respond to suchrapid changes.

FIG. 1 is a diagram illustrating the performance of an existing projectcollaboration method. Referring to FIG. 1 , such a UI/UX developmentsystem is applied by reflecting the development of technology andcustomer needs to newly develop or upgrade programs. In a process ofdeveloping a project, opinions are presented through meetings, and whenagreement on a configuration of the project is reached throughdiscussion, a function definition is prepared based on agreed content,and an information architecture (IA), a wireframe, a storyboard, and adesign task are performed in order to perform publishing, and through anoperation of developing programs, corresponding content is coded to opena service.

There are problems in that the UI/UX development system has a structurein which, in a project progress process, only when each of operationsfrom agreement to programming development should be completed, a nextoperation may be performed, and since data in each operation is observedindependently, task resources and time requirements are inevitablyincurred.

Therefore, there is a need to improve a process in order to reduce suchtask resources and time requirements.

DISCLOSURE Technical Problem

Therefore, in order to solve the above problems, the present disclosureis directed to provide a method capable of concurrently agreeing on aplurality of operations and collectively performing the operations bymaking a function definition, an information architecture (IA), awireframe, a storyboard, and a design, etc. into UX-bits that can beeasily selected, applied, and combined.

The present disclosure is also directed to provide an automatic designgenerating artificial neural network device and method whichautomatically outputs a design when elements are input in order toreduce a development period and development costs.

Technical Solution

Hereinafter, specific means for achieving the object of the presentdisclosure will be described.

In order to solve the above problems, a development projectcollaboration method according to one embodiment of the presentdisclosure includes receiving a project goal, receiving detailed tasksfor achieving the project goal, arranging content agreed upon in theproject goal and the detailed tasks into UX-bits in an integratedfunction definition, combining the UX-bits based on the functiondefinition and arranging information architecture (IA) elements,constituting a wireframe based on the IA elements, designing aconfiguration page based on the wireframe, providing a verification pagefor verifying the configuration page formed by combining the UX-bits,and when a modification occurs in at least one of the project goal, thedetailed task, the UX-bit, the IA element, the wireframe, and thedesign, simultaneously modifying the project goal, the detailed task,the UX-bit, the IA element, the wireframe, and the design to which themodification is reflected in the entire project.

According to one embodiment of the present disclosure, the UX-bits mayinclude function UX-bits and design UX-bits of which levels areclassified into a plurality of levels.

According to one embodiment of the present disclosure, levels of thefunction UX-bit may be classified into level 1 including a word unit,level 2 including a word combination, level 3 including a group ofconcepts, level 4 including hierarchy information, level 5 includingarea information, level 6 including screen information, level 7including a code, and level 8 including a completed service.

According to one embodiment of the present disclosure, levels of thedesign UX-bit may include level 1 including a formative attribute, level2 including a value of a formative attribute, level 3 including acombination of a formative attribute and a value, level 4 including agroup of attribute combinations, level 5 including a design language,level 6 including screen information, and level 7 including publishinginformation.

According to one embodiment of the present disclosure, the arranging ofthe IA elements may include presenting at least one IA option based onthe function definition and providing an edit page for editing aposition and a connection relationship of the IA elements.

According to one embodiment of the present disclosure, the providing ofthe edit page may include arranging the dragged and dropped IA elements,connecting the arranged IA elements with lines according to levels orhierarchies, and modifying the arranged and connected IA elementsaccording to a practitioner's choice.

According to one embodiment of the present disclosure, the constitutingof the wireframe may include presenting a plurality of wireframe layoutsbased on the arranged and connected IA elements, and selecting andcustomizing the wireframe layouts by reflecting a practitioner's choice.

According to one embodiment of the present disclosure, the constitutingof the wireframe may include reflecting a change in arrangement of theIA elements, which is caused by selecting and customizing a wireframelayout, on the arranged and connected IA elements.

According to one embodiment of the present disclosure, the designing ofthe configuration page may include presenting a design option in orderof design UX-bit levels, and modifying the design option by reflecting apractitioner's choice.

According to one embodiment of the present disclosure, the providing ofthe verification page formed by combining the UX-bits may includeapplying and outputting a design to the selected and customizedwireframe layout on the screen, and performing a simulation in the sameenvironment as an actual user environment.

According to one embodiment of the present disclosure, the performing ofthe simulation in the same environment as the actual user environmentmay include outputting a controller on at least a portion of the screen,and implementing driving of the project according to an input of thepractitioner input to the controller.

In order to solve the above problems, a development projectcollaboration program using a UX-bit according to one embodiment of thepresent disclosure includes commands that perform a development projectcollaboration method using a UX-bit which controls a screen defined byan operating system at predetermined intervals. The development projectcollaboration method using a UX-bit may include receiving a projectgoal, receiving detailed tasks for achieving the project goal, arrangingcontent agreed upon in the project goal and the detailed tasks intoUX-bits in an integrated function definition, combining the UX-bitsbased on the function definition and arranging IA elements, constitutinga wireframe based on the IA elements, designing a configuration pagebased on the wireframe, providing a verification page for verifying theconfiguration page formed by combining the UX-bits, and when amodification occurs in at least one of the project goal, the detailedtask, the UX-bit, the IA element, the wireframe, and the design,simultaneously modifying the project goal, the detailed task, theUX-bit, the IA element, the wireframe, and the design to which themodification is reflected in the entire project.

An object of the present disclosure may be achieved by providing anautomatic design generating artificial neural network device using auser experience (UX)-bit including an image theme encoding module thatis an encoding module which receives image theme data, which is an imagerepresenting a theme of a web/app graphic design to be generated by apractitioner, as input data, and outputs an image theme encoding vectoras output data, a text theme encoding module that is an encoding modulewhich receives text theme data, which is text representing the theme ofthe web/app graphic design to be generated by the practitioner, as inputdata, and outputs a text theme encoding vector as output data, a UX-bitgeneration module that is a module which receives the image themeencoding vector and the text theme encoding vector as input data andoutputs UX-bit attributes of a plurality of UX elements as output data,a design generation module that is an upsampling artificial neuralnetwork module which receives the image theme encoding vector, the texttheme encoding vector, and the UX-bit attribute as input data andoutputs design data meaning a specific web/app graphic design as outputdata, an image theme discriminator that is a module used in a learningsession of the design generation module and is a pre-learned artificialneural network module which, when the design data and the image themedata are input as input data, outputs an image theme discriminationvector, which means a probability of similarity between the design dataand the image theme data, as output data, a text theme discriminatorthat is a module used in the learning session of the design generationmodule and is a pre-learned artificial neural network module which, whena design encoding vector that is an encoding vector of the design dataand the text theme encoding vector are input as input data, outputs atext theme discrimination vector, which means a probability ofsimilarity between the design encoding vector and the text themeencoding vector, as output data, and a UX-bit attribute discriminatorthat is a module used in the learning session of the design generationmodule and is a pre-learned artificial neural network module which, whenthe design encoding vector and an encoding vector of the UX-bitattribute are input as input data, outputs a UX-bit attributediscrimination vector, which means a probability of similarity betweenthe design encoding vector and the encoding vector of the UX-bitattribute, as output data, wherein, in the learning session of thedesign generation module, parameters of the design generation module areupdated in a direction in which a representative design loss, which iscomposed of a difference between the design data and web/app designreference data (ground truth) in which similarity with the image themeencoding vector and similarity with the text theme encoding vector aregreater than or equal to a specific level in an 3encoding vector of apre-stored web/app design corresponding thereto, an image themediscrimination loss including the image theme discrimination vector, atext theme discrimination loss including the text theme discriminationvector, and a UX-bit attribute determination loss including the UX-bitattribute discrimination are reduced.

The automatic design generating artificial neural network device mayfurther include a nonlinear network which is connected to the designgeneration module, is a network having a nonlinear structure in which aplurality of fully connected (FC) layers are consecutively connected,and is a module which receives a concatenated vector obtained byconcatenating the image theme encoding vector and the text themeencoding vector as input data, outputs a theme vector as output data,and inputs the output theme vector to a plurality of layers in a networkof the design generation module for each scale, wherein, in the designgeneration module, the UX-bit attribute is input as input data, and anoise vector is input to each layer having a scale to which the themevector is input.

The automatic design generating artificial neural network device mayfurther include a common theme segment module which is connected to theimage theme encoding module and is a module which receives the imagetheme data as input data and outputs common theme segment data as outputdata, wherein, in a learning session of the common theme segment module,parameters of the common theme segment module are updated in a directionin which similarity between the text theme encoding vector and the imagetheme encoding vector are increased.

The UX-bit attributes may include UX-bit function attributes and UX-bitdesign attributes, and the UX-bit generation module may include a UXelement generation module that is a module which generates the pluralityof the UX elements to match the UX-bit function attribute and the UX-bitdesign attribute with a specific UX element, and an recurrent neuralnetwork (RNN) module that is an artificial neural network module whichoutputs the UX-bit function attribute and the UX-bit design attributefor the UX element.

The UX-bit attributes may include UX-bit function attributes and UX-bitdesign attributes, and the UX-bit generation module may include a UXelement generation module that is a module which generates the pluralityof the UX elements to match the UX-bit function attribute and the UX-bitdesign attribute with a specific UX element, and an RNN module that isan artificial neural network module which outputs the UX-bit functionattribute and the UX-bit design attribute for the UX element, whereinthe RNN module includes RNN blocks including a first RNN cell and asecond RNN cell as a basic unit, the image theme encoding vector and thetext theme encoding vector are used as initial input data, the first RNNcell receives the initial input data or output data of a previous celland RNN hidden layer information and outputs the UX-bit functionattribute of an nth UX element, and the second RNN cell receives theUX-bit function attribute, which is the output data of the previouscell, and the RNN hidden layer information, and outputs the UX-bitdesign attribute for the UX-bit function attribute of the nth UX elementoutput from the first RNN cell.

The UX-bit attributes may include UX-bit function attributes and UX-bitdesign attributes, and the UX-bit generation module may include a UXelement generation module that is a module which generates the pluralityof the UX elements to match the UX-bit function attribute and the UX-bitdesign attribute with a specific UX element, an RNN module that is anartificial neural network module which outputs the UX-bit functionattribute and the UX-bit design attribute for the UX element, and areinforcement learning module which is configured such that the UX-bitattributes of all pre-generated UX elements, the image theme encodingvector, and the text theme encoding vector are input as an environment,an RNN block of the RNN module is set as an agent, a situation, in whichan n_(th) UX element having the UX-bit function attributes and theUX-bit design attributes is virtually included in the UX-bit functionattributes and the UX-bit design attributes of first to (n−1)^(th)elements, is set as a state, in such a state, the UX-bit functionattributes and the UX-bit design attributes output for the n^(th) UXelement by the RNN block, which is the agent, are input for an action,and as similarity is high between comparison information and the UX-bitfunction attributes and the UX-bit design attributes of the nth UXelement which are output data, a relatively high reward is generated toupdate a hidden layer of the RNN block which is the agent, wherein thecomparison information means a concatenation between the image themeencoding vector and the text theme encoding vector.

Another object of the present disclosure may be achieved by providing anautomatic design generating artificial neural network method using aUX-bit including an image theme encoding operation of, by an image themeencoding module, receiving image theme data, which is an imagerepresenting a theme of a web/app graphic design to be generated by apractitioner, as input data, and outputting an image theme encodingvector as output data, a text theme encoding operation of, by a texttheme encoding module, receiving text theme data, which is textrepresenting the theme of the web/app graphic design to be generated bythe practitioner, as input data, and outputting a text theme encodingvector as output data, a UX-bit generating operation of, by a UX-bitgeneration module, receiving the image theme encoding vector and thetext theme encoding vector as input data and outputting UX-bitattributes of a plurality of UX elements as output data, and a designgenerating operation of, by a design generation module, receiving theimage theme encoding vector, the text theme encoding vector, and theUX-bit attribute as input data and outputting design data meaning aspecific web/app graphic design as output data, wherein, in a learningsession of the design generation module, parameters of the designgeneration module are updated in a direction in which a representativedesign loss, which is composed of a difference between the design dataand web/app design reference data (ground truth) in which similaritywith the image theme encoding vector and similarity with the text themeencoding vector are greater than or equal to a specific level in apre-stored web/app design corresponding thereto, an image themediscrimination loss including a discrimination difference between theimage theme discrimination vector and the design data, a text themediscrimination loss including a discrimination difference between thetext theme discrimination vector and the design data, and a UX-bitattribute discrimination loss including a discrimination differencebetween the design data and a UX-bit attribute discrimination vectorand, which is a discrimination vector for the UX-bit attributes arereduced.

Still another object of the present disclosure may be achieved byproviding an automatic design generating artificial neural networksystem including a practitioner client which receives image theme datathat is an image representing a theme of a web/app graphic design to begenerated by a practitioner and text theme data that is textrepresenting the theme to be generated by the practitioner from thepractitioner, and the automatic design generating artificial neuralnetwork device using a UX-bit which receives the image theme data andthe text theme data from the practitioner client and outputs design datacorresponding to the image theme data and the text theme data.

Advantageous Effects

As described above, the present disclosure has the following effects.

First, according to one embodiment of the present disclosure, with a newunit called a UX-bit, a plurality of operations to be performed step bystep in a development project may be collectively agreed, and resultsare derived, thereby considerably reducing task resources and timerequirements.

Second, according to one embodiment of the present disclosure, in aprocess of maintaining a project, by using the UX-bit, collectivemaintenance is performed rather than sequential maintenance, therebyreducing task resources and time requirements.

DESCRIPTION OF DRAWINGS

The following drawings attached to this specification illustrateexemplary embodiments of the present disclosure and function tofacilitate further understanding of the technical spirit of the presentdisclosure along with the detailed description of the invention.Accordingly, the present disclosure should not be construed as beinglimited to only matters illustrated in the drawings:

FIG. 1 is a diagram illustrating the performance of an existing projectcollaboration method;

FIG. 2 is a flowchart of a development project collaboration methodusing a user experience (UX)-bit according to one embodiment of thepresent disclosure;

FIGS. 3 and 4 are diagrams for describing a concept of a UX-bit of adevelopment project collaboration method using a UX-bit according to oneembodiment of the present disclosure;

FIG. 5 is a diagram illustrating properties and level classification ofa UX-bit of a development project collaboration method using a UX-bitaccording to one embodiment of the present disclosure;

FIGS. 6 to 13 are diagrams for describing a method in which a system ofa development project collaboration method using a UX-bit processes dataaccording to one embodiment of the present disclosure;

FIG. 14 is a diagram for describing a concept of an informationarchitecture (IA);

FIG. 15 shows diagrams for describing an IA arrangement and connectionpage of a development project collaboration method using a UX-bitaccording to one embodiment of the present disclosure;

FIG. 16 shows diagrams for describing a wireframe configuration page ofa development project collaboration method using a UX-bit according toone embodiment of the present disclosure;

FIGS. 17 and 18 are diagrams for describing a verification page of adevelopment project collaboration method using a UX-bit according to oneembodiment of the present disclosure;

FIG. 19 is a schematic diagram illustrating an operational relationshipof an automatic design generating artificial neural network deviceaccording to one embodiment of the present disclosure;

FIG. 20 is a schematic diagram illustrating a specific configuration ofthe automatic design generating artificial neural network deviceaccording to one embodiment of the present disclosure;

FIGS. 21 and 22 are exemplary diagrams of a ConvNet (convolutionalneural network (CNN) encoder) included in an image theme encoding module(10) and a text theme encoding module (11) according to one embodimentof the present disclosure;

FIG. 23 is a schematic diagram illustrating a specific configuration ofa UX-bit generation module (12) according to one embodiment of thepresent disclosure;

FIG. 24 is a schematic diagram illustrating a UX-bit element generationmodule (120) according to one embodiment of the present disclosure;

FIG. 25 is a schematic diagram illustrating a recurrent neural network(RNN) module (12) according to one embodiment of the present disclosure;

FIG. 26 is a schematic diagram illustrating a reinforcement learningmodule (122) according to one embodiment of the present disclosure;

FIG. 27 is a schematic diagram illustrating a reinforcement learningmodule (122) according to a modified example of the present disclosure;

FIG. 28 is a flowchart illustrating an operation example of areinforcement learning module (122) according to one embodiment of thepresent disclosure;

FIG. 29 is a schematic diagram illustrating automatic generation ofUX-bit attributes of a UX-bit generation module (12) according to oneembodiment of the present disclosure;

FIG. 30 is a schematic diagram illustrating a the structure of a designgeneration module (15) according to one embodiment of the presentdisclosure;

FIG. 31 is a schematic diagram illustrating an operational relationshipof a design generation module (15) according to another embodiment ofthe present disclosure;

FIG. 32 is a schematic diagram illustrating a skip connection of adesign generation module (15) according to one embodiment of the presentdisclosure; and

FIG. 33 is a schematic diagram illustrating a common theme segmentmodule (101) according to another embodiment of the present disclosure.

MODES OF THE INVENTION

Hereinafter, embodiments of the present disclosure will be described indetail with reference to the accompanying drawings so that those skilledin the art may easily practice the present disclosure. In describing anoperational principle relating to the embodiments of the presentdisclosure, however, when a detailed description of relevant functionsor constructions is deemed to make the subject matter of the presentdisclosure unnecessarily obscure, the detailed description will beomitted.

Furthermore, the same reference numerals designate elements havingsimilar functions and operations throughout the drawings. Throughout thespecification, when it is described that one specific element isconnected to the other element, the specific one element may be directlyconnected to the other element or indirectly connected to the otherelement through a third element. Furthermore, when it is described thatspecific element includes another element, it means that the specificelement does not exclude another element, but may include otherelements, unless otherwise described.

A computing device described in the present specification may includeall media in which images are expressed through interaction with a user(only information is not transferred), and examples thereof a digitaltelevision (TV), a desktop computer, a mobile phone, a smartphone, atablet personal computer (PC), a laptop computer, a digital broadcastingterminal, a personal digital assistant (PDA), a portable multimediaplayer (PMP), a navigation device, a head mounted display (HMD), anaugmented reality (AR) card, a head-up display (HUD), and the like.

An operating system (OS) refers to a program that controls hardware andsoftware of a computing device to allow a practitioner to use thecomputing device. An OS can manage computer resources such asprocessors, storage devices, and input/output interface devices whileserving as an interface between hardware and application programs. Forexample, types of the OS may include Android, iOS, Windows, Mac, Tizen,Unix, and Linux.

An application program (hereinafter referred to as “program”) refers tosoftware developed such that a practitioner can perform a specific taskusing a device. For example, there may be an e-mail program, a messengerprogram, a schedule management program, and a document editing program.In addition, the program may include instructions necessary to perform aspecific task. Here, the instructions constituting the program may bedifferent from each other according to a type of an OS.

A screen may be defined by an OS. The screen may be a virtualtwo-dimensional area having coordinates within a preset range. Thescreen may be displayed by a display device, and a practitioner mayvisually recognize the screen through the display device. A coordinaterange of the screen may be adjusted according to the availableresolution of the display device by the OS. A coordinate unit of thescreen may correspond to a position of a pixel of the display device.

Some programs may be formed such that an operation form is displayed asan object on a screen. For example, an operation form of some programsmay be displayed in the form of an “execution window” as an object on ascreen. For example, the execution window may include a document editingwindow outputted as a document editing program is executed and a webbrowser window outputted as a web browser application is executed. Asanother example, the operation form of some programs may be displayed asan object on the screen in the form of a “mouse cursor (a pointer thatvisually moves along with the movement of a mouse or touchpad andgenerally has a shape of an arrow).” For example, a mouse cursor and atouch point may be displayed to move within a screen in response to aninput means being detected by a practitioner. Some other programs may beformed to operate in the form of a background without being separatelydisplayed on a screen.

When a plurality of objects are displayed on a screen, objectsrepresenting operation forms of the programs may be displayed accordingto a preset level. For example, in an OS, when an area displaying afirst object corresponding to a first program overlaps an areadisplaying a second object corresponding to a second program on ascreen, in an overlapping area, the first object may be set to bedisplayed prior to the second object.

Development Project Collaboration Method Using User Experience (UX)-Bit

A development project collaboration method using a UX-bit according toan embodiment of the present disclosure may be performed by a computingdevice or by the computing device executing a program.

Hereinafter, embodiments of the present disclosure will be described indetail with reference to the accompanying drawings. In the drawings, thesame or similar reference numerals are used to refer to the same orsimilar elements.

FIGS. 3 and 4 are diagrams for describing a concept of a UX-bit of adevelopment project collaboration method using a UX-bit according to oneembodiment of the present disclosure.

Referring to FIGS. 3 and 4 , the UX-bit may be defined as a smallestunit constituting a UX in a process of developing a project.

The UX-bit may be classified and tagged according to a related subjector category. The UX-bit may be classified according to a tagged subjector category and stored in a database. The UX-bit may be combined toother UX-bits to form one output. A computing device may visualize theoutput to which the UX-bit is coupled and output the visualized outputon a screen.

The UX-bit may include information classified according to operations ofa function definition, an information architecture (IA), a wireframe, adesign, and programming. The UX-bit is the smallest unit of elementsconstituting the UX and may include a color, a position, a layout, andthe like.

In the development project collaboration method using a UX-bit accordingto one embodiment of the present disclosure, step-by-step operations maybe performed in parallel on the UX-bit to check the output at once.

The function definition is information in which, when a service is newlymade or renewed, what function is dealt with in a project, what apurpose is, what data is used, and what technical issue is required arelisted. The function definition may include information about a functionincluded in the project, a method of implementing the function, and amethod of operating the function. The function definition may include atask scenario for each function, an expected function when a service isconstructed, requirements, a detailed function, and an application area.

The IA is structural information entirely showing what structure aservice has, what function the service has, what role the service plays,and what screen the service is displayed on. The IA may facilitate aninteraction between a practitioner and a service to allow thepractitioner to easily and quickly find desired information and mayinclude structural information necessary to match an overall image of aservice that is thought by a planner, a designer, and a practitioner.

The IA may include depth, labeling, and flow.

The wireframe may approximately summarize and show content, a function,a UI element, and a layout displayed on representative screens before avisual design operation and may refer to a screen blueprint.

The wireframe may include information showing how to convert afunctional structure into a function of an interface in order to set adirection of a user interface (UI)/UX design.

The wireframe may include a size and an arrangement of a logo, a menu,an image, text, and a button for each area.

The design may include a format that may be produced as a UX designresult such as a photoshop document (PSD), an XD document, a schema, ora zeplin that visually shows a screen to be actually implemented byadding design elements (color, typography, and image) based on thewireframe.

The design may include information capable of supplementing visualproblems by confirming a planned service with an actual visual designbefore development.

The design may include actual screen data for each function including anaction.

The programming may include code work information for implementing avisually planned design into an actual service. In the programming, itis possible to implement a planned service and review a technology. Theprogramming may include a service development language and codesimplemented with the service development language.

FIG. 5 is a diagram illustrating properties and level classification ofa UX-bit of a development project collaboration method using a UX-bitaccording to one embodiment of the present disclosure.

Referring to FIG. 5 , the UX-bit may be classified into a functionUX-bit and a design UX-bit according to properties. The UX-bit may beclassified into a plurality of levels according to each property.

Levels of the function UX-bit may be classified into level 1 including aword unit, level 2 including a word combination, level 3 including agroup of concepts, level 4 including hierarchy information, level 5including area information, level 6 including screen information, level7 including a code, and level 8 including a completed service.

The word unit of level 1 of the function UX-bit may include wordinformation about a concept of a function. The word combination of level2 may include a function represented by a complex concept (compoundword: word+word). In level 3, the group of concepts may includeinformation in which concepts are grouped and classified according to acertain criterion. Level 4 may include information about a hierarchybetween the concepts or the groups. Level 5 may include area informationaccording to a concept, a group, and a hierarchy. Level 6 may includewireframe information including an area, a target, and a goal which aredisplayed on a screen. Level 7 may include a code for implementing thefunction. Level 8 may include a service completed with a combination ofimplemented functions.

Levels of the design UX-bit may include level 1 including a formativeattribute, level 2 including a value of a formative attribute, level 3including a combination of a formative attribute and a value, level 4including a group of attribute combinations, level 5 including a designlanguage, level 6 including screen information, and level 7 includingpublishing information.

The formative attribute of level 1 of the design UX-bit may includetext, image, color, shape, and size information. The value of theformative attribute of level 2 may include a text font, a color, a size,and a thickness. The group of the attribute combinations of level 4 mayshow a visually displayed style. The design language of level 5 mayinclude concept and atmosphere information. The screen information oflevel 6 may include format that may be produced as a UX design resultsuch as a PSD, an XD document, a schema, or a zeplin. The publishinginformation of level 7 may include hypertext mark-up language (HTML) andcascading style sheet (CSS) information.

According to one embodiment of the present disclosure, when a searchfunction is generated in the function UX-bit, a UX-bit related to asearch may be reviewed at level 1, a UX-bit related to an automaticsearch may be reviewed at level 2, a range of the search function may bereviewed at level 3, a hierarchy of a search and a result may bedetermined at level 4, a position of a search and a search result may bedetermined at level 5, a wireframe of a search and a search result maybe determined at level 6, a programming language for a search functionmay be reviewed at level 7, and a completed search function may begenerated at level 8.

According to one embodiment of the present disclosure, when text isinput to a title area of a page in the design UX-bit, text content maybe reviewed at level 1, a font, a color, a size, and a thickness may bereviewed at level 2, a title may be reviews at level 3, a form may bereviewed at level 4, a detailed design feeling may be reviewed at level5, a PSD file to which design elements are applied may be reviewed atlevel 6, and a publishing code for converting a PSD format may bereviewed and may be coupled to a result of level 7 of the functionUX-bit to generate a completed search function at level 7.

FIG. 2 is a flowchart of a development project collaboration methodusing a UX-bit according to one embodiment of the present disclosure.FIGS. 6 to 13 are diagrams for describing a method in which a system ofa development project collaboration method using a UX-bit processes dataaccording to one embodiment of the present disclosure.

Referring to FIGS. 2 to 10 , the development project collaborationmethod using a UX-bit according to the embodiment of the presentdisclosure may include operation S110 of receiving a project goal,operation S120 of receiving detailed tasks for achieving the projectgoal, and operation S130 of arranging content agreed upon in the projectgoal and the detailed tasks into UX-bits in an integrated functiondefinition.

In operation S110, a computing device may receive the project goal froma practitioner. In operation S110, the project goal may includeresources such as a project target value, a project period, inputmanpower, and input infrastructure equipment.

In operation S120, the computing device may receive the detailed tasksfor achieving the project goal from the practitioner. When there are aplurality of project goals, the computing device may receive detailedtasks for each goal.

In operation 120, an execution plan UX-bit database (DB) may receive thedetailed tasks for achieving the project goal input by the practitioner.

In operation S130, the computing device may arrange the content agreedupon in the project goal and the detailed tasks into the UX-bits in theintegrated function definition.

In operation S130, by using data stored in at least one of the executionplan UX-bit DB and a function definition UX-bit DB, the computing devicemay arrange the content agreed upon in the project goal and the detailedtasks into the UX-bits.

FIG. 14 is a diagram for describing a concept of an IA.

Referring to FIG. 14 , depths of an IA structure may be visuallydistinguished, a position at which any information is positioned bybeing labeled in the structure may be identified, and the IA may beidentified by indicating a flow of each information with an arrow.

The development project collaboration method using a UX-bit according tothe embodiment of the present disclosure may include operation S140 ofcombining the UX-bits based on the function definition and arranging IAelements.

In operation S140, the computing device may extract information afunction of a program to be developed, and IA, wireframe, design, andprogramming information from the function definition determined by thepractitioners. The computing device may arrange the IA elements based oncontent defined in the function definition.

In operation S140, by using data stored in at least one of the functiondefinition UX-bit DB and an IA UX-bit DB, the computing device mayarrange the IA element at a position determined in the functiondefinition.

The computing device may arrange the IA elements by searching for theUX-bit using a word extracted from the function definition as a keyword.

The computing device may modify the arrangement of the IA elements byreflecting results input by the practitioner through drag and drop.

In the development project collaboration method using a UX-bit accordingto one embodiment of the present disclosure, the computing device mayreceive a word from the practitioner or from the function definition.The computing device may find a keyword related to the word input by thepractitioner or extracted from the function definition. The computingdevice may search for and provide a keyword similar to or related to theword. The computing device may label and store the keyword. The keywordmay be labeled with a similar or related word. When a labeled word isinput as a search word, the keyword may be exposed as a keyword that maybe selected by the practitioner. The keyword may include relevance asprobability information according to a ratio in which the labeled wordis input as the search word, exposed, and selected. The practitioner mayselect a keyword displayed on a screen of the computing device.

Operation S140 may include an operation of presenting at least one IAoption based on the function definition and an operation of providing anedit page for editing a position and a connection relationship of the IAelements.

The operation of providing the edit page may include an operation ofarranging the dragged and dropped IA elements, an operation ofconnecting the arranged IA elements with lines according to levels orhierarchies, an operation of modifying the arranged and connected IAelements according to a practitioner's choice, and an operation ofreflecting a change in arrangement of the IA elements, which is causedby selecting and customizing a wireframe layout, on the arranged andconnected IA elements.

FIG. 15 shows diagrams for describing an IA arrangement and connectionpage of the development project collaboration method using a UX-bitaccording to one embodiment of the present disclosure.

Referring to FIG. 15 , the practitioner may generate an IA by draggingand dropping the UX-bits on the IA arrangement and connection page. Whenthe practitioner moves the IA elements through drag and drop to generatethe IA, according to information such as a hierarchy of the IA elements,the IA elements may be connected to form a flow and complete the IA.When pressing and holds the IA element in the completed IA, thepractitioner may modify, delete, and copy the IA element.

The development project collaboration method using a UX-bit according tothe embodiment of the present disclosure may include operation S150 ofconstituting a wireframe based on the determined IA elements.

In operation S150, the computing device may constitute the wireframebased on the IA elements arranged in operation S140.

Operation S150 of constituting the wireframe may include an operation ofpresenting a plurality of wireframe layouts based on the arranged andconnected IA elements and an operation of selecting and customizing thewireframe layouts by reflecting a practitioner's choice.

In operation S150, by using data stored in at least one of the functiondefinition UX-bit DB, the IA UX-bit DB, and a wireframe UX-bit DB, thecomputing device may constitute the wireframe according to thearrangement of the IA elements.

FIG. 16 shows diagrams for describing a wireframe configuration page ofthe development project collaboration method using a UX-bit according toone embodiment of the present disclosure.

Referring to FIG. 16 , in the development project collaboration methodusing a UX-bit according to one embodiment of the present disclosure, awireframe layout may be presented based on an IA determined in aprevious operation.

In the presented wireframe layout, the wireframe layout may be changedby reflecting a practitioner's choice and custom content. Thepractitioner may change the wireframe layout through a simple operationof dragging or clicking the layout. In the operation of selecting andcustomizing the wireframe layout by reflecting the practitioner'schoice, wireframe layouts of a plurality of screens (units) may bechanged. According to one embodiment of the present disclosure, when thewireframe layout is modified and changed for each screen (unit) and onescreen (unit) is modified and changed, it is possible to move to a pagefor modifying and changing a next screen (unit) connected to the IA.

In the development project collaboration method using a UX-bit accordingto one embodiment of the present disclosure, the computing device maydetect a keyword related to a project. The computing device may searchfor and present U-bits based on the extracted keyword. UX-bits selectedfrom among the found UX-bits by the practitioner may be transferred tothe IA arrangement and connection page.

The development project collaboration method using a UX-bit according tothe embodiment of the present disclosure may include operation S160 ofdesigning a configuration page based on the determined wireframe.

In operation S160, the design UX-bits selected by the practitioner maybe combined based on the IA arrangement and connection page to designthe configuration page.

Operation S160 of designing the configuration page may include anoperation of presenting a design option in order of design UX-bitlevels, an operation of modifying the design option by reflecting apractitioner's choice, and an operation of storing and managing theoption of the design UX-bits generated as publishing UX-bits.

Operation S160 may further include an operation of reviewingconnectivity between the stored publishing UX-bit and a developmentlanguage.

In operation S160, the computing device may constitute a design of theconstituted wireframe using data stored in at least one of the functiondefinition UX-bit DB, the UX-bit IA DB, the wireframe UX-bit DB, and adesign UX-bit DB.

Operation S160 may include the operation of presenting the design optionin order of the design UX-bit levels, then operation of modifying thedesign option by reflecting the practitioner's choice, and the operationof storing and managing the option of the design UX-bits generated asthe publishing UX-bits.

Referring to FIGS. 8 and 9 , in the operation of storing and managingthe option of the design UX-bits generated as the publishing UX-bits,the design of the wireframe constituted using data stored in at leastone of the function definition UX-bit DB, the IA UX-bit DB, thewireframe UX-bit DB, and the design UX-bit DB may be constituted andstored in the publishing UX-bit DB.

The development project collaboration method using a UX-bit according tothe embodiment of the present disclosure may include operation S170 ofproviding a verification page for verifying the configuration pageformed by combining the UX-bits.

In operation S170, the computing device may provide a page for verifyinga screen formed by combining the UX-bits combined on the configurationpage.

Operation S170 may include an operation of applying and outputting adesign to the selected and customized wireframe layout on the screen andan operation of performing a simulation in the same environment as anactual user environment.

The operation of performing the simulation in the same environment asthe actual user environment may include an operation of outputting acontroller on at least a portion of the screen and an operation ofimplementing driving of the project according to an input of thepractitioner input to the controller.

When the practitioner touches the screen, the controller may be outputon at least a portion of the screen.

The display of the controller is merely an example of the case ofT-commerce, and depending on the situation, a mouse cursor, a mobilescreen configuration, or the like may be output to perform thesimulation in the same environment as the actual user environment.

FIGS. 17 and 18 are diagrams for describing a verification page of thedevelopment project collaboration method using a UX-bit according to oneembodiment of the present disclosure.

Referring to FIGS. 17 and 18 , a service page to which a design isapplied based on the wireframe determined in a previous operation forthe practitioner may be displayed on a screen. The practitioner maymanipulate the controller on the service page to check whether an actualprogram operates normally.

The development project collaboration method using a UX-bit according tothe embodiment of the present disclosure may include operation S180 of,when a modification occurs in at least one of the project goal, thedetailed task, the UX-bit, the IA element, the wireframe, and thedesign, simultaneously modifying the project goal, the detailed task,the UX-bit, the IA element, the wireframe, and the design to which themodification is reflected in the entire project.

In operation S180, when, through an input of the practitioner, themodification occurs in at least one of the project goal, the detailedtask, the UX-bit, the IA element, the wireframe, and the design, thecomputing device may simultaneously modify the project goal, thedetailed task, the UX-bit, the IA element, the wireframe, and the designto which the modification is reflected in the entire project.

In operation S180, when a modification occurs in the project using datastored in at least one of the function definition UX-bit DB, the IAUX-bit DB, the wireframe UX-bit DB, the design UX-bit DB, and thepublishing UX-bit DB, the computing device may collectively modify thefunction definition, the IA, the wireframe, the design and a publishingpart.

A development project collaboration program using a UX-bit according toone embodiment of the present disclosure includes commands that performa development project collaboration method using a UX-bit which controlsa screen defined by an OS at predetermined intervals. The developmentproject collaboration method using a UX-bit may include receiving aproject goal, receiving detailed tasks for achieving the project goal,arranging content agreed upon in the project goal and the detailed tasksinto UX-bits in an integrated function definition, combining the UX-bitsbased on the function definition and arranging IA elements, constitutinga wireframe based on the IA elements, designing a configuration pagebased on the wireframe, providing a verification page for verifying theconfiguration page formed by combining the UX-bits, and when amodification occurs in at least one of the project goal, the detailedtask, the UX-bit, the IA element, the wireframe, and the design,simultaneously modifying the project goal, the detailed task, theUX-bit, the IA element, the wireframe, and the design to which themodification is reflected in the entire project.

Automatic design generating artificial neural network device and methodusing UX-bit

FIG. 19 is a schematic diagram illustrating an operational relationshipof an automatic design generating artificial neural network deviceaccording to one embodiment of the present disclosure. FIG. 20 is aschematic diagram illustrating a specific configuration of the automaticdesign generating artificial neural network device according to oneembodiment of the present disclosure. As shown in FIGS. 19 and 20 , theautomatic design generating artificial neural network device 1 accordingto one embodiment of the present disclosure is configured to receiveimage theme data 100 and text theme data 200 as input data and outputdesign data 300 as output data to a practitioner client.

The image theme data 100 may be an image representing a theme of aweb/app graphic design to be generated by a practitioner through theautomatic design generating artificial neural network device 1 and maybe input to the automatic design generating artificial neural networkdevice 1 from the practitioner client by a practitioner's choice.

The text theme data 200 may be text representing the theme of theweb/app graphic design to be generated by the practitioner through theautomatic design generating artificial neural network device 1 and maybe input to the automatic design generating artificial neural networkdevice 1 from the practitioner client by an input of the practitioner.

The design data 300 refers to a web/app graphic design for a specificpage generated by the automatic design generating artificial neuralnetwork device 1.

An image theme encoding module 10 is an encoding module which receivesthe image theme data 100 as input data and outputs an image themeencoding vector as output data. The image theme encoding module 10according to one embodiment of the present disclosure may include aConvNet (convolutional neural network (CNN) encoder).

A text theme encoding module 11 is an encoding module which receives thetext theme data 200 as input data and outputs a text theme encodingvector as output data. For example, the text theme encoding module 11according to one embodiment of the present disclosure may segment thetext theme data 200 in units of a phoneme and uses the text theme data200 as input data and may refer to an encoding module composed of anartificial neural network having a structure of single layer convolutionwith ReLU-max pooling with stride 5=segment embeddings-four layerhighway network-single layer bidirectional GRU.

FIGS. 21 and 22 are exemplary diagrams of a ConvNet (CNN encoder)included in the image theme encoding module 10 and the text themeencoding module 11 according to one embodiment of the presentdisclosure. As shown in FIGS. 21 and 22 , as an example, a simpleConvNet may be constructed as [INPUT-CONV-RELU-POOL-FC]. In the case ofan input vector, when an input matrix INPUT has a width of 32, a lengthof 32, and a red-green-blue (RGB) channel, an input size may be[32×32×3]. A CONV layer (Cony. Filter) is connected to a partial area ofthe input matrix and calculates the dot product of the connected areaand a weight thereof. A result volume has a size of [32×32×12]. A RELUlayer is an activation function such as a max (0,x) applied to eachelement. The RELU layer does not change the size of [32×32×12] of thevolume. As a result, the RELU layer generates activation map 1. Apooling (POOL) layer performs downsampling on a “horizontal/vertical”dimension and outputs a reduced volume (activation map 2) such as[16×16×12]. The fully connected (FC) layer calculates class scores andoutputs a pressure distribution vector volume having a size of [1×1×n](output layer). The FC layer is connected to all elements of a previousvolume.

As described above, the ConvNet included in the image theme encodingmodule 10 and the text theme encoding module 11 transforms an originalmatrix composed of pixel values into class scores for distributionthrough each layer. Some layers have parameters, but some layers do nothave parameters. In particular, the CONV/FC layers are activationfunctions that include not only input volumes but also weights andbiases. On the other hand, the RELU/POOL layers are fixed functions.Parameters of the CONV/FC layers are learned through a gradient descentsuch that a class score for each matrix is equal to a label of acorresponding matrix.

The parameters of the CONV layer of the ConvNet included in the imagetheme encoding module 10 and the text theme encoding module 11 arecomposed of a series of trainable filters. Each filter is small in ahorizontal/vertical dimension but encompasses an entire depth in a depthdimension. During a forward pass, each filter is slid along ahorizontal/vertical dimension of an input volume (exactly, is convolved)to generate a two-dimensional activation map. When a filter is slid overan input, a dot product is performed between the filter and the inputvolume. Through such a process, the ConvNet learns a filter thatactivates for a specific pattern at a specific position in input data.Such activation maps are stacked in a depth dimension to become anoutput volume. Therefore, each element of the output volume handles onlya small area of the input, and neurons in the same activation map sharethe same parameters because the same filter is applied.

Examples of network structures usable for the ConvNet included in theimage theme encoding module 10 and the text theme encoding module 11 areas follows.

LeNet. The first successful ConvNet applications have been created byYann LeCun in the 1990s. Among the first successful ConvNetapplications, a LeNet architecture for reading zip codes or numbers isthe most famous.

AlexNet. An AlexNet created by Alex Krizhevsky, Ilya Sutskever, andGeoff Hinton makes a ConvNet famous in the field of computer vision. TheAlexNet participated in the ImageNet ILSVRC challenge 2012 and won firstplace by a large margin over second place (top 5: error rate of 16% andsecond place: error rate of 26%). An architecture thereof is basicallysimilar to that of the LeNet but is deeper and larger. Also, in thepast, unlike the POOL layer being stacked immediately after one CONVlayer, a plurality of CONV layers were stacked.

ZF Net. The winner of the ILSVRC 2013 were created by Matthew Zeiler andRob Fergus. The winner is called a ZFNet after the authors. The ZFNetwas created by modifying hyperparameters such as resizing a middle CONVlayer in the AlexNet.

GoogLeNet. The winners of ILSVRC 2014 are Szegedy et al. A GoogLeNet wascreated by Google. The biggest contribution of such a model is topropose an Inception module which considerably reduces the number ofparameters (4 M as compared with 60 M for the AlexNet). In addition,instead of FC layers, average pooling is used at the end of a ConvNet toreduce a lot of parameters that are not very important.

VGGNet. A network that won second place in the ILSVRC 2014 was a modelcalled a VGGNet created by Karen Simonyan and Andrew Zisserman. Thebiggest contribution of the model is to show that a depth of a networkis a very important factor for good performance. Among a plurality ofmodels proposed by Karen Simonyan and Andrew Zisserman, the best modelincludes 16 CONV/FC layers, and all convolutions are 3×3, and allpooling is only 2×2. Although the VGGNet had slightly lower matrixclassification performance than the GoogLeNet, it was later found thatthe VGGNetto had better performance on several transfer learning tasks.Thus, recently, the VGGNet has been used for extracting matrix features.The VGGNet has disadvantages in that a lot of memory (140M) is used, anda large amount of computation is required.

ResNet. A Residual Network created by Kaiming He et al. won the ILSVRC2015. The Residual Network is characterized by using a unique structurecalled a skip connection and using a lot of batch normalization. Such anarchitecture does not use an FC layer in a last layer.

In particular, the image theme encoding module 10 may include anartificial neural network having the following structure.

-   -   [55×55×96] CONV1: 96@ 11×11, stride=4, parameter=0    -   [27×27×96] MAX POOL1: 3×3, stride=2    -   [27×27×256] CONV2: 256@ 5×5, stride=1, parameter=2    -   [13×13×256] MAX POOL2: 3×3, stride=2    -   [13×13×384] CONV3: 384@ 3×3, stride=1, parameter=1    -   [13×13×384] CONV4: 384@ 3×3, stride=1, parameter=1    -   [13×13×256] CONV5: 256@ 3×3, stride=1, parameter=1    -   [6×6×256] MAX POOL3: 3×3, stride=2    -   FC6: 4096 neurons    -   FC7: 4096 neurons

In the example above, CONV refers to a convolution layer, MAX POOLrefers to a pooling layer, and FC refers to a fully connected layer.

A UX-bit generation module 12 is a module which receives an image themeencoding vector and a text theme encoding vector as input data andoutputs UX-bit function attributes and UX-bit design attributes ofUX-bits for a design, which is to be generated in the automatic designgenerating artificial neural network device 1, as output data. FIG. 23is a schematic diagram illustrating a specific configuration of theUX-bit generation module 12 according to one embodiment of the presentdisclosure. As shown in FIG. 23 , the UX-bit generation module 12 mayinclude a UX element generation module 120, a recurrent neural network(RNN) module 121, and a reinforcement learning module 122.

The UX element generation module 120 is a module which generates aplurality of UX elements to match the UX-bit function attribute and theUX-bit design attribute with a specific UX element. More specifically,the UX element generation module 120 is a module which generates theplurality of UX elements to match a combination of the UX-bit functionattributes (hereinafter referred as UX-bit function attributes) and acombination of the UX-bit design attributes (hereinafter referred asUX-bit design attributes) with the specific UX element. FIG. 24 is aschematic diagram illustrating the UX-bit generation module 120according to one embodiment of the present disclosure. As shown in FIG.24 , the UX element generation module 120 may be configured to generatea plurality of UX elements to match UX-bit function attributes/UX-bitdesign attributes with a specific UX element and may be configured tocontinuously generate new UX elements in which at least one UX-bitattribute is selected from a plurality of UX-bit attributes by the RNNmodule 121 to match the specific UX element.

The RNN module 121 may be an artificial neural network module includinga plurality of RNN blocks which match a combination of UX-bit functionattributes and a combination of UX-bit design attributes (hereinafterreferred to as UX-bit design attributes) with a pre-generated UXelement, wherein the plurality of RNN blocks include a first RNN celland a second RNN cell. An image theme encoding vector and a text themeencoding vector may be received as initial input data, the first RNNcell may receive the initial input data or output data of a previouscell and RNN hidden layer information and may output UX-bit functionattributes (a specific combination of the UX-bit function attributes) ofan n^(th) UX element, and the second RNN cell may be configured toreceive the UX-bit function attributes and the RNN hidden layerinformation, which are output data of the previous cell, and outputUX-bit design attributes (a specific combination of the UX-bit designattributes) with respect to the UX-bit function attributes of the n^(th)UX element output from the first RNN cell.

FIG. 25 is a schematic diagram illustrating the RNN module 12 accordingto one embodiment of the present disclosure. As shown in FIG. 25 , theRNN module 121 according to one embodiment of the present disclosure mayinclude a plurality of RNN blocks, and one RNN block may include a firstRNN cell and a second RNN cell. As shown in FIG. 7 , the first RNN cellmay output UX-bit function attributes of an n^(th) UX element, and thesecond RNN cell may be configured to output UX-bit design attributeswith respect to the UX-bit function attributes of the correspondingn^(th) UX element. When the UX-bit function attribute output from thefirst RNN cell is “end,” the number of UX elements is preset, and thepreset number of UX elements is reached, the RNN module 121 according toone embodiment of the present disclosure may end inference when rewardsfor all actions calculated by the reinforcement learning module 122 arenegative numbers. According to the RNN module 121 according to oneembodiment of the present disclosure, UX-bit function attributes andUX-bit design attributes are sequentially output according to UXelements, and thus attributes of a UX element generated in a previousstep affect attributes of a UX element to be generated in a next step,thereby obtaining an effect in which functions do not entirely overlapeach other, and a design allows a unified UX element to be generated.

The reinforcement learning module 122 which trains the RNN module may beprovided such that UX-bit attributes of all pre-generated UX elements,an image theme encoding vector, and a text theme encoding vector areinput as an environment, each RNN block of the RNN module 121 is set asan agent, a situation, in which an n^(th) UX element having specificUX-bit function attributes and UX-bit design attributes is virtuallyincluded in UX-bit function attributes and UX-bit design attributes offirst to (n−1)^(th) elements, is set as a state, in such a state, UX-bitfunction attributes and UX-bit design attributes output for the n^(th)UX element by the RNN block, which is the agent, are input for anaction, and as similarity (for example, cosine similarity) is highbetween comparison information and UX-bit function attributes and UX-bitdesign attributes of a UX element (n^(th) UX element) of a current step,which are output data of the current step, or a divergence (for example,a Kullback-Leibler divergence) is small therebetween, a high reward isgenerated to update a hidden layer of the RNN block which is the agent.FIG. 26 is a schematic diagram illustrating the reinforcement learningmodule 122 according to one embodiment of the present disclosure. Asshown in FIG. 26 , the reinforcement learning module 122 may be providedsuch that UX-bit attributes of all pre-generated UX elements, an imagetheme encoding vector, and a text theme encoding vector are input as anenvironment, in a state of a situation in which the RNN block that is anagent virtually includes an n^(th) UX element having specific UX-bitfunction attributes and UX-bit design attributes in UX-bit functionattributes and UX-bit design attributes of first to (n−1)^(th) elements,an action, which outputs UX-bit function attributes and UX-bit designattributes for the n^(th) UX element, is performed, and as similarity(for example, cosine similarity) is high between comparison information(concatenation between an image theme encoding vector and a text themeencoding vector) and UX-bit function attributes and UX-bit designattributes of a UX element (n^(th) UX element) of a current step, whichis output data of the current step, or a divergence (for example, aKullback-Leibler divergence) is small therebetween, a high reward isgenerated to update a hidden layer of the RNN block which is the agent.The RNN block optimized by the reinforcement learning module 122according to one embodiment of the present disclosure may be providedsuch that the hidden layer is fixed.

Thus, there is an effect in which optimal UX-bit attributescorresponding to an image theme encoding vector and a text themeencoding vector are generated to be customized for each UX element bythe UX element generation module 120 and the RNN module 121. Inaddition, without the need to consider the number of all cases of UX-bitfunction attributes and UX-bit design attributes of all UX elements thatcan be generated by the UX element generation module 120, thereinforcement learning module 122 is configured to be sequentiallyoptimized for respective UX elements, thereby obtaining an effect inwhich the number of cases to be calculated by the reinforcement learningmodule 122 is reduced to reduce computing resources.

A reinforcement learning module 122 according to a modified example ofthe present disclosure may be provided such that an RNN block is updatedthrough more effective reinforcement learning by the followingconfiguration. FIG. 27 is a schematic diagram illustrating thereinforcement learning module 122 according to the modified example ofthe present disclosure. As shown in FIG. 27 , the reinforcement learningmodule 122 according to the modified example of the present disclosuremay include a value network 211 which is an artificial neural networkconfigured to learn a value function for outputting a value in aspecific state and a policy network 210 which learns a policy functionfor outputting a probability of each of UX-bit function attributes andUX-bit design attributes. The policy network 210 and the value network211 according to the modified example of the present disclosure may beconnected to a specific RNN block of the RNN module 121. The policynetwork 210 and the value network 211 may be connected to the RNN blockto output UX-bit attributes for a specific UX element.

The policy network 210 is an artificial neural network which determinesa probability of UX-bit function attributes and UX-bit design attributesselected in each state of the reinforcement learning module 122. Thepolicy network 210 learns a policy function to output a probability ofthe selected UX-bit function attributes and UX-bit design attributes. Acost function of the policy network may be a function obtained bycalculating cross entropy by multiplying a policy function by a costfunction of the value network and then taking a policy gradient. Forexample, the cost function of the policy network may be formed as inEquation 1 below. The policy network may be back propagated based on theproduct of the cross entropy and a time difference error which is thecost function of the value network.

−∇_(θ)log π_(θ)(a _(i) |s _(i))(r _(i+1) +γV _(w)(s _(i+1))−V _(w)(s_(i)))  [Equation 1]

In Equation 1, π may be denote a policy function, Θ may denote a policynetwork parameter, πΘ(a_(i) |s_(i)) may denote a probability of aspecific action being performed (on UX-bit function attributes andUX-bit design attributes) in a current episode, V may denote a valuefunction, w may denote a value network parameter, s_(i) may denote stateinformation of a current episode i, S_(i+1) may denote state informationof a next episode i+1, r_(i+1) may denote a reward expected to beobtained in the next episode, V_(w)(s_(i)) may denote reward possibilityin the current episode, V_(w)(s_(i+1)) may denote reward possibility inthe next episode, and y may denote a depreciation rate. In this case,r_(i+1) may be configured to receive similarity between comparisoninformation (concatenation between an image theme encoding vector and atext theme encoding vector) and UX-bit function attributes and UX-bitdesign attributes of a UX element of a current step.

Before reinforcement learning is performed, based on UX-bit functionattributes and UX-bit design attributes of a previous UX element andperformance information according thereto (similarity between comparisoninformation (concatenation between an image theme encoding vector and atext theme encoding vector) and UX-bit function attributes and UX-bitdesign attributes of a UX element of a current step), the policy network210 according to one embodiment of the present disclosure learn thebasics of a policy by a weight of the policy network being updatedthrough supervised learning. That is, the weight of the policy networkmay be set through supervised learning based on the UX-bit functionattributes and the UX-bit design attributes of the previous UX element,and the performance information. Accordingly, the policy network can bevery quickly trained by a history of the UX-bit function attributes andthe UX-bit design attributes of the previous UX element.

In addition, according to one embodiment of the present disclosure,during supervised learning of the policy network 210, the supervisedlearning may be performed based on calculation part type information andparameter information of a previous layer and performance informationaccording thereto in addition to a random vector. For the random vector,for example, a Gaussian distribution may be used. Accordingly, there isan effect in which the policy network can output challenging UX-bitfunction attributes and UX-bit design attributes with a randomprobability. When, during the supervised learning of the policy network210, the supervised learning is performed based on the UX-bit functionattributes and the UX-bit design attributes of the previous UX element,and the performance information according thereto, the selection ofUX-bit function attributes and UX-bit design attributes of a UX elementresults in being optimized within a policy of the previous UX element.However, when the random vector is included in the supervised learningof the policy network according to one embodiment of the presentdisclosure, there is an effect in which, as reinforcement learning isperformed, the policy network can learn more effective UX-bit functionattributes and UX-bit design attributes than the policy of the previousUX element.

The value network 211 is an artificial neural network which derives apossibility, in which a reward is achieved in each state to which thereinforcement learning module 122 may be changed, and learns a valuefunction. The value network 211 provides a direction in which the RNNblock, which is an agent, is to be updated. To this end, an inputvariable of the value network 211 is set as state information which isinformation about a state of the reinforcement learning module 122, andan output variable of the value network 211 may be set as rewardpossibility information which is a possibility of the RNN blockachieving a reward (similarity between comparison information(concatenation between an image theme encoding vector and a text themeencoding vector) and UX-bit function attributes and UX-bit designattributes of a UX element of a current step). The reward possibilityinformation according to one embodiment of the present disclosure may becalculated by a Q-function as in Equation below.

Q _(π)(s,a)=E _(π) [R _(t+1) +γR _(t+2) + . . . |S _(t) =s,A _(t)=a]  [Equation 2]

In Equation 2 above, Q_(π) may denote entire reward possibilityinformation expected in the future in a case in which a state is s andan action is a in a specific policy π, R may denote a reward for aspecific period, and gamma γ may denote a depreciation rate. S_(t) maydenotes a state at a time t, A_(t) may denote an action at the time t,and E may denote an expected value. Reward possibility information (Qvalue) according to one embodiment of the present disclosure defines anupdate direction and size of the policy network 210.

In this case, a cost function of the value network may be a mean squareerror (MSE) function for a value function and for example, may be formedas in Equation 3 below. The value network 211 may be back propagatedbased on a time difference error which is the cost function of the valuenetwork.

(r _(i+1) +γV _(w)(s _(i+1))−V _(w)(s _(i)))²  [Equation 3]

In Equation 3, V may denote a value function, w may denote a valuenetwork parameter, s_(i) may denote state information of a currentepisode i, S_(i+1) may denote state information of a next episode i+1,r_(i+1) may denote a reward expected to be obtained in the next episode,V_(w)(s_(i)) may denote reward possibility in the current episode,V_(w)(s_(i+1)) may denote reward possibility in the next episode, and ymay denote a depreciation rate. In this case, r_(i+1) may be configuredto receive similarity between comparison information (concatenationbetween an image theme encoding vector and a text theme encoding vector)and UX-bit function attributes and UX-bit design attributes of a UXelement of a current step.

Accordingly, when a state of the reinforcement learning module 122 ischanged, the value network may be updated in a direction in which agradient of the cost function of Equation 3 descends.

According to one embodiment of the present disclosure, while the valuenetwork is trained separately from the policy network, a Q value of thevalue network is supervised instead of starting randomly, and thus thereis an effect in which rapid learning is possible. Accordingly, there isan effect in which it is possible to greatly reduce a burden ofexploration in an action of selecting a combination of UX-bit functionattributes and UX-bit design attributes having very high complexity.

According to the reinforcement learning module 122 according to oneembodiment of the present disclosure, when the policy network 210 thathas performed supervised learning selects UX-bit function attributes andUX-bit design attributes of a current episode i, the value network 211is trained to predict a reward (similarity between comparisoninformation (concatenation between an image theme encoding vector and atext theme encoding vector) and UX-bit function attributes and UX-bitdesign attributes of a UX element of a current step) when the selectedUX-bit function attributes and UX-bit design attributes proceed. Thepolicy network 210 and the value network 211 of the reinforcementlearning module 122 which have been trained are combined with asimulation using the RNN block and finally used to select UX-bitfunction attributes and UX-bit design attributes.

In addition, according to the value network 211 according to oneembodiment of the present disclosure, there is an effect in which anupdate of the policy network that outputs a probability of selectedUX-bit function attributes and UX-bit design attribute may be performedevery episode. In existing reinforcement learning, there is a problem inthat an update of a reinforcement learning model is performed after allepisodes are finished, and thus it has been difficult to apply theexisting reinforcement learning to an RNN module that sequentiallygenerates UX-bit function attributes and UX-bit design attributes.

The RNN block is a component which searches for optimal UX-bit functionattributes and UX-bit design attributes by performing a plurality ofsimulations on various states and various actions based on a pluralityof agents calculated in the policy network and the value network. TheRNN block according to one embodiment of the present disclosure mayutilize, for example, a Monte Carlo tree search and has a structure inwhich each node in a tree represents a state, each edge represents avalue expected according to a specific action on a corresponding state,and while a current state is set as a root node, a leaf node is expandedwhenever a new action is performed to transition to a new state. In theRNN block according to one embodiment of the present disclosure, whenoptimal UX-bit function attributes and UX-bit design attributes aresearched for using the Monte Carlo tree search, the search may beprocessed through four operations of selection, expansion, evaluation,and backup operations.

The selection operation of the RNN block is an operation in which anaction having the highest value among selectable actions is selected andperformed until the leaf node expands from the current state. In thiscase, a value of a value function stored in the edge and a visitfrequency value for balancing exploration and use are used. An equationfor selecting an action in the selection operation is as follows.

a _(t) =armax_(a)(Q(s _(t) ,a)+u(s _(t) ,a))  [Equation 4]

In Equation 4 above, at denotes an action at a time t (action ofselecting UX-bit function attributes and UX-bit design attributes),Q(s_(t),a) denotes a value of a value function stored in a tree, andu(s_(t),a) denotes a value inversely proportional to the number ofvisits of a state-action pair and is used to balance exploration anduse.

The expansion operation of the RNN block is an operation of, when asimulation proceeds to a leaf node, adding a new node as a leaf node byacting according to a probability of the policy network trained throughsupervised learning.

The evaluation operation of the RNN block is an operation of evaluatinga value of a leaf node through a value (reward possibility) determinedfrom the newly added leaf node using the value network and a rewardobtained by proceeding until an episode of selecting UX-bit functionattributes and UX-bit design attributes from a leaf node using thepolicy network is ended. An equation below is an example of evaluating avalue of a new leaf node.

V(s _(L))=(1−λ)v _(θ)(S _(L))+λz _(L)  [Equation 5]

In Equation 5 above, V(s_(L)) may denote a value of a leaf node, λ maydenote a mixing parameter, v_(θ)(s_(L)) may denote a value obtainedthrough the value network, and z_(L) may denote a reward obtained bycontinuing a simulation.

The backup operation of the RNN block is an operation of reevaluating avalue of nodes visited during a simulation by reflecting the value ofthe newly added leaf node and updating a visit frequency. An equationbelow is an example of reevaluating a value and updating a visitfrequency.

$\begin{matrix}{{{N\left( {s,a} \right)} = {\sum\limits_{i}{1\left( {s,a,i} \right)}}}{{Q\left( {s,a} \right)} = {\frac{1}{N\left( {s,a} \right)}{\sum\limits_{i}{1\left( {s,a,i} \right){V\left( S_{L}^{i} \right)}}}}}} & \left\lbrack {{Equation}6} \right\rbrack\end{matrix}$

In Equation 6 above, s^(i) _(L) may denote a leaf node in an i^(th)simulation, and 1(s,a,i) may denote whether an edge (s,a) is visited inthe i^(th) simulation. When a tree search is completed, an algorithm maybe configured to select the most visited edge (s,a) from a root node.According to the RNN block according to one embodiment of the presentdisclosure, there is an effect in which a plurality of simulations canbe performed in advance on a plurality of UX-bit function attributes andUX-bit design attributes selected by the policy network based on thevalue network to select optimal UX-bit function attributes and UX-bitdesign attributes.

According to one embodiment of the present disclosure, the reinforcementlearning module 122 may be provided such that a plurality of agents areprovided. There is an effect in which, when the plurality of agents areprovided, UX-bit function attributes and UX-bit design attributesselected by the reinforcement learning module 122 may compete with eachother for a specific state and each of specific UX-bit functionattributes and UX-bit design attributes to select the most optimalUX-bit function attributes and UX-bit design attributes.

FIG. 28 is a flowchart illustrating an operation example of thereinforcement learning module 122 according to one embodiment of thepresent disclosure. As shown in FIG. 28 , when a state s(t) is input bythe UX element generation module 120, various UX-bit function attributesand UX-bit design attributes are input to the RNN block by a pluralityof agents of the policy network 210 through the value network 211, andUX-bit function attributes and UX-bit design attributes are selected bya probability a(t) of selected UX-bit function attributes and UX-bitdesign attributes which is an action output by the RNN block, an episodet is ends, and an episode t+1 starts. In the episode t+1, a state changes(t+1) by a(t) is input again by the UX element generation module 120,and a reward r(t+1) according to a(t) is immediately input to update thevalue network 211 and the policy network 210.

Regarding an operation example of the UX-bit generation module 12, FIG.29 is a schematic diagram illustrating automatic generation of UX-bitattributes of the UX-bit generation module 12 according to oneembodiment of the present disclosure. As shown in FIG. 29 , for example,one RNN block may be provided with respect to one UX element, an imagetheme encoding vector and a text theme encoding vector may be input to afirst RNN cell of a first RNN block, UX-bit function 1 among UX-bitfunction attributes may be output for a first UX element, previouslyoutput UX-bit function 1 may be input to a second RNN cell of the firstRNN block, UX-bit design 1 among UX-bit design attributes may be outputfor the first UX element to generate UX-bit attributes for the first UXelement, UX-bit design 1 previously output for the first UX element maybe input to a first RNN cell of a second RNN block, UX-bit function 2among the UX-bit function attributes may be output for the second UXelement, previously output UX-bit function 2 may be input to a secondRNN cell of the second RNN block, and UX-bit design 2 among UX-bitdesign attributes may be output to generate UX-bit attributes for thesecond UX element.

An IA generation module 13 is a module which generates an IA based onUX-bit attributes (UX-bit function attributes and UX-bit designattributes) of a plurality of UX elements generated by the UX-bitgeneration module 12. The IA generated by the IA generating module 13may be generated as shown in FIGS. 14 and 15 .

A wireframe generation module 14 is a module which generates a wireframebased on UX-bit attributes (UX-bit function attributes and UX-bit designattributes) of a plurality of UX elements generated by the UX-bitgeneration module 12. The wireframe generated by the wireframegeneration module 14 may be generated as shown in FIG. 16 .

A design generation module 15 is a module which outputs design data asoutput data using an image theme encoding vector, a text theme encodingvector, and X-bit attributes (UX-bit function attributes and UX-bitdesign attributes) of a plurality of UX elements generated by the UX-bitgeneration module 12 as input data. The design data generated by thedesign generation module 15 may refer to a graphic design of a web/apppage or the like in which design elements are formed in a wireframe.

Regarding a specific configuration of the design generation module 15,FIG. 30 is a schematic diagram illustrating a structure of the designgeneration module 15 according to one embodiment of the presentdisclosure. As shown in FIG. 30 , the design generation module 15 may becomposed of an artificial neural network in which a concatenated vectorobtained by concatenating an image theme encoding vector, a text themeencoding vector, and UX-bit attribute data by being connected to theimage theme encoding module 10, the text theme encoding module 11 andthe UX-bit generation module 12 are input as input data of the designgeneration module 15, design data is output as output data, and in alearning session, parameters are updated by an image theme discriminator150, a text theme discriminator 151, and a UX-bit attributediscriminator 152.

Regarding an overall embodiment of the image theme encoding module 10,the text theme encoding module 11, and the design generation module 15,for example, the image theme encoding module 10 according to oneembodiment of the present disclosure may be composed of a ConvNetincluding a plurality of consecutive convolution layers, pooling layers,and fully connected layers that receive image theme data standardized ina specific dimension as input data and encode an image theme encodingvector, which is a latent variable of 1×1×k, as output data. Inaddition, the image theme encoding module 10 may form a skip connectionstructure with the design generation module 15. The text theme encodingmodule 11 may be composed of a ConvNet including a plurality ofconsecutive convolution layers, pooling layers, and fully connectedlayers that divide and input text theme data based on phonemes andencode a text theme encoding vector, which is a latent variable of1×1×k, as output data. In addition, the text theme encoding module 11may form a skip connection structure with the design generation module15.

In a learning session of the image theme encoding module 10, image themedata input to the image theme encoding module 10 according to oneembodiment of the present disclosure may be input in a channel-wiseconcatenation structure for each convolution layer of the image themeencoding module 10 and the design generation module 15. In this case,due to a configuration in which the image theme data is input in thechannel-wise concatenation structure for each convolution layer of theimage theme encoding module 10 and the design generation module 15, in alearning session of the image theme encoding module 10 and the designgeneration module 15, a vanishing gradient is improved, featurepropagation is strengthened, and the number of parameters is saved,thereby obtaining an effect of reducing computing resources.

In a learning session of the design generation module 15, parameters ofthe design generation module 15 may be updated in a direction in which arepresentative design loss composed of a difference between design dataand corresponding reference data (ground truth) is reduced. Arepresentative design loss, which is one of loss functions of the designgeneration module 15, may also include a mean square loss, a crossentropy loss, and the like and for example, may be as follows.

$\begin{matrix}{L_{c} = {\frac{1}{N}{\underset{i = 1}{\sum\limits^{N}}{{L_{g} - L_{G}}}_{2}}}} & \left\lbrack {{Equation}7} \right\rbrack\end{matrix}$

In Equation above, L_(c) may denotes a representative design loss, N maybe the number of pairs of design data and reference data (batch size), imay denote a specific image pair among N pairs of design data andreference data, L_(g) may denote the design data, L_(G) may denote thereference data as a ground truth, and ∥ ∥₂ may denote L₂-norm. In thiscase, the reference data in the representative design loss may refer toa web/app design reference of which similarity with an image themeencoding vector and similarity with a text theme encoding vector aregreater than or equal to a specific level among pre-stored encodingvectors of a web/app design.

The image theme discriminator 150 is a module used in a learning sessionof the design generation module 15 and an artificial neural networkmodule which is pre-trained to receive design data output from thedesign generation module 15 and image theme data that is a reference(ground truth) and output an image theme discrimination vector fordistinguishing between the image theme data and the design data. Theimage theme discriminator 150 according to one embodiment of the presentdisclosure may include a CONCAT function and a plurality of convolutionlayers. The image theme discriminator 150 may be pre-trained separatelyfrom the design generation module 15, may be used in a learning sessionof the design generation module 15 in a state of being pre-trained(state in which parameters are fixed), and may be configured to outputan image theme discrimination vector in an image theme discriminationloss included in a loss function of the design generation module 15 inthe learning session of the design generation module 15.

In a learning session of the image theme discriminator 150, the learningsession may have a configuration in which (real-labeled) image themedata and (fake-labeled) design data output from the design generationmodule 15 are input to the image theme discriminator 150, the imagetheme discriminator 150 outputs an image theme discrimination vector fordistinguishing between real and fake of the design data, wherein theimage theme discrimination vector may include a real class and a fakeclass or include only a real class, and parameters of a ConvNet of theimage theme discriminator 150 are updated in a direction in which a lossincluding a difference between the image theme discrimination vector andan actual (real or fake) label of the design data is reduced.

That is, in a learning session of the image theme discriminator 150, thelearning session may have a configuration in which parameters of theimage theme discriminator 150 are updated such that an image themediscrimination vector Di(x,y) in a distribution of design data in whichimage theme data is set as a reference becomes minimum (0), and an imagetheme discrimination vector Di(y,y) in an image theme data distribution,which is a ground truth distribution corresponding thereto, becomesmaximum (1). For example, a loss included in a loss function of theimage theme discriminator 150 may be as follows.

L _(i) =E _(x˜L) _(g) _(,y˜L) _(G) [log(1−D _(i)(x,y)]+E _(y˜L) _(G)[log(D _(i)(y,y)]  [Equation 8]

In Equation above, L_(i) denotes a loss of the image themediscriminator, x denotes design data, y denotes image theme data that isreference data as a ground truth, y˜L_(G) denotes a distribution of thereference data, x˜L_(g) denotes a distribution of the design data, andD_(i)(x,y) denotes an image theme discrimination vector (probabilityvalue between 0 and 1) output from the image theme discriminator 150when the design data is input using the image theme data as a reference.D_(i)(y,y) denotes an image theme discrimination vector (probabilityvalue between 0 and 1) output from the image theme discriminator 150when the image theme data is input as a reference. In a learning sessionof the image theme discriminator 150, when it is determined that thedesign data output by the design generation module 15 is a design thatis not similar to the image theme data, the image theme discriminator150, the loss of the image theme discriminator is output such thatD_(i)(x,y) is close to 0, and L_(i) is close to 0 (minimum value), andwhen it is determined that the design data is a design that is similarto the image theme data, the loss of the image theme discriminator isoutput such that D_(i)(x,y) is close to 1, and L_(i) is close to ∞(maximum value), which may be applied to learning (parameter update) ofthe image theme discriminator 150. In addition, parameters of the imagetheme discriminator 150 may be updated in a direction in whichD_(i)(y,y) approaches 1. Regarding a learning session of the image themediscriminator 150, the learning session of the image theme discriminator150 may have a configuration in which learning data labeled assynthesized in the design generation module 15 and learning data labeledas not synthesized are provided as leaning data of the image themediscriminator 150, the learning data is input to the image themediscriminator 150 to output an image theme discrimination vector, a lossof the learning data is calculated based on the image themediscrimination vector, and parameters of the image theme discriminator150 are updated in a direction in which the calculated loss of the imagetheme discriminator is minimized.

Regarding an operational relationship of the image theme discriminator150 in a learning session of the design generation module 15, a lossfunction of the design generation module 15 may include an image themediscrimination vector output when design data is input to the imagetheme discriminator 150 as input data. In the loss function of thedesign generation module 15, for example, an image theme discriminationvector may be provided as a loss (hereinafter referred to as image themedetermination loss) as follows.

L _(di) =E _(x˜L) _(g) _(,y˜L) _(G) [log(D _(i)(x,y)]  [Equation 9]

In Equation above, L_(i) denotes an image theme discrimination loss, xdenotes design data, y denotes image theme data, y˜L_(G) denotes adistribution of reference data, and D_(i)(x,y) denotes an image themediscrimination vector (probability value between 0 and 1) output fromthe image theme discriminator 150 when the design data is input usingthe image theme data as a reference. In a learning session of the designgeneration module 15 performed by the image theme discriminator 150, thelearning session may have a configuration in which the image themediscrimination loss including the image theme discrimination vector isincluded in a loss function of the design generation module 15, andparameters of the design generation module 15 are updated such that theimage theme discrimination vector D_(i)(x,y) in a distribution of thedesign data becomes maximum (1) and thus L_(di) becomes 0 (minimumvalue) (such that it is determined that the design data is a design thatis similar to the image theme data).

The text theme discriminator 151 is a module used in a learning sessionof the design generation module 15 and an artificial neural networkmodule which is pre-trained to receive design data output from thedesign generation module 15 and text theme data that is a reference(ground truth) and output a text theme discrimination vector fordistinguishing between the text theme data and the design data. The texttheme discriminator 151 according to one embodiment of the presentdisclosure may include a CONCAT function and a plurality of convolutionlayers. The text theme discriminator 151 may be pre-trained separatelyfrom the design generation module 15, may be used in a learning sessionof the design generation module 15 in a state of being pre-trained(state in which parameters are fixed), and may be configured to output atext theme discrimination vector in a text theme discrimination lossincluded in a loss function of the design generation module 15 in thelearning session of the design generation module 15.

In a learning session of the text theme discriminator 151, the learningsession may have a configuration in which a (real-labeled) text themeencoding vector that is an encoding vector of text theme data and a(fake-labeled) design encoding data that is an encoding vector of designdata output from the design generation module 15 are input to the texttheme discriminator 151, the text theme discriminator 151 outputs a texttheme discrimination vector for distinguishing between real and fake ofthe design data (distinguishing between the text theme data and thedesign data), wherein the text theme discrimination vector may include areal class and a fake class or include only a real class, and parametersof a ConvNet of the text theme discriminator 151 are updated in adirection in which a loss including a difference between the text themediscrimination vector and an actual (real or fake) label of the designdata is reduced.

That is, in a learning session of the text theme discriminator 151, thelearning session may have a configuration in which parameters of thetext theme discriminator 151 are updated such that a text themediscrimination vector D_(i)(x,z) in a distribution of design data inwhich a text theme encoding vector is set as a reference becomes minimum(0), and a text theme discrimination vector D_(i)(z,z) in a distributionof a text theme encoding vector of text theme data, which is a groundtruth distribution corresponding thereto, becomes maximum (1). Forexample, a loss included in a loss function of the text themediscriminator 151 may be as follows.

L _(t) =E _(x˜L) _(g) _(,z˜L) _(G) [log(1−D _(t)(x,z)]+E _(z˜L) _(G)[log(D _(t)(z,z)]  [Equation 10]

In Equation above, L_(i) denotes a loss of the text theme discriminator,x denotes an encoding vector of design data, z denotes a text themeencoding vector that is reference data as a ground truth, z˜L_(G)denotes a distribution of the reference data, x˜L_(g) denotes adistribution of the encoding vector of the design data, and D_(t)(x,z)denotes a text theme discrimination vector (probability value between 0and 1) output from the text theme discriminator 151 when the encodingvector of the design data is input using the text theme encoding vectoras a reference. D_(t)(z,z) denotes a text theme discrimination vector(probability value between 0 and 1) output from the text themediscriminator 151 when the text theme encoding vector is input as areference. In a learning session of the text theme discriminator 151,when it is determined that the encoding vector of the design data outputby the design generation module 15 is not similar to the text themeencoding vector which is an encoding vector of text theme data, the lossof the text theme discriminator is output such that D_(t)(x,z) is closeto 0, and L_(t) is close to 0 (minimum value), and when it is determinedthat the encoding vector of the design data is similar to the themeencoding vector, the loss of the text theme discriminator is output suchthat D_(i)(x,z) is close to 1, and L_(t) is close to ∞ (maximum value),which may be applied to learning (parameter update) of the text themediscriminator 151. In addition, parameters of the text themediscriminator 151 may be updated in a direction in which D_(t)(z,z)approaches 1. Regarding a learning session of the text themediscriminator 151, the learning session of the text theme discriminator151 may have a configuration in which learning data labeled assynthesized in the design generation module 15 and learning data labeledas not synthesized are provided as leaning data of the text themediscriminator 151, the learning data is input to the text themediscriminator 151 to output a text theme discrimination vector, a lossof the learning data is calculated based on the text themediscrimination vector, and parameters of the text theme discriminator151 are updated in a direction in which the calculated loss of the texttheme discriminator is minimized.

Alternatively, regarding a modified example of a loss of the text themediscriminator 151, a loss included in a loss function of the text themediscriminator 151 may also include a mean square loss, a cross entropyloss, and the like. For example, when a binary cross entropy loss isapplied, the loss may be as follows.

$\begin{matrix}{L_{t} = {\frac{1}{N}{\sum\limits_{i = 1}^{N}{- {\log\left( \frac{v_{i} \cdot s_{i}}{{v_{i}}_{2} \cdot {s_{i}}_{2}} \right)}}}}} & \left\lbrack {{Equation}11} \right\rbrack\end{matrix}$

In Equation above, L_(t) may denote a loss of the text themediscriminator, N may be the number of pairs of text theme encodingvectors and design encoding vectors (batch size), i may denote aspecific pair of N pairs of text theme encoding vectors and designencoding vectors, and v may denote the design encoding vector, s maydenote the text theme encoding vector, and ∥ ∥₂ may denote L₂-norm

Regarding an operational relationship of the text theme discriminator151 in a learning session of the design generation module 15, a lossfunction of the design generation module 15 may include a text themediscrimination vector output when a text theme encoding vector is inputto the text theme discriminator 151 as input data. In the loss functionof the design generation module 15, for example, the text themediscrimination vector may be provided as a loss (hereinafter referred toas text theme determination loss) as follows.

L _(dt) =E _(x˜L) _(g) _(,z˜L) _(G) [log(D _(t)(x,z)]  [Equation 12]

In Equation above, L_(dt) denotes a text theme discrimination loss, xdenotes an encoding vector of design data, x˜L_(g) denotes adistribution of the encoding vector of the design data, z denotes a texttheme encoding vector, z˜L_(G) denotes a distribution of the text themeencoding vector, and D_(t)(x,z) denotes a text theme discriminationvector (probability value between 0 and 1) output from the text themediscriminator 150 when the encoding vector of the design data is inputusing the text theme encoding vector as a reference. In a learningsession of the design generation module 15 performed by the text themediscriminator 151, the learning session may have a configuration inwhich the text theme discrimination loss including the text themediscrimination vector is included in a loss function of the designgeneration module 15, and parameters of the design generation module 15are updated such that the text theme discrimination vector D_(t)(x,z) inthe distribution of the encoding vector of the design data becomesmaximum (1) and thus L_(dt) becomes 0 (minimum value) (such that it isdetermined that the encoding vector of the design data is similar to thetext theme encoding vector).

Regarding the UX-bit attribute discriminator 152, the UX-bit attributediscriminator 152 is a module used in a learning session of the designgeneration module 15 and is an artificial neural network module which ispre-trained to receive design data output from the design generationmodule 15 and UX-bit attribute data (reference as a ground truth) outputfrom the UX-bit generation module 12 and output a UX-bit attributediscrimination vector for distinguishing between the design data and theUX-bit attribute data. The UX-bit attribute discriminator 152 accordingto one embodiment of the present disclosure may include a CONCATfunction and a plurality of convolution layers. The UX-bit attributediscriminator 152 may be pre-trained separately from the designgeneration module 15, may be used in a learning session of the designgeneration module 15 in a state of being pre-trained (state in whichparameters are fixed), and may be configured to output a UX-bitattribute discrimination vector in a UX-bit discrimination loss includedin a loss function of the design generation module 15 in the learningsession of the design generation module 15.

In a learning session of the UX-bit attribute discriminator 152, thelearning session may have a configuration in which a (real-labeled)encoding vector of UX-bit attribute data and a (fake-labeled) designencoding vector that is a design encoding vector of design data outputfrom the design generation module 15 are input to the UX-bit attributediscriminator 152, the UX-bit attribute discriminator 152 outputs aUX-bit attribute discrimination vector for distinguishing between realdata and fake of the design data, wherein the UX-bit attributediscrimination vector may include a real class and a fake class orinclude only a real class, and parameters of a ConvNet of the UX-bitattribute discriminator 152 are updated in a direction in which a lossincluding a difference between the UX-bit attribute discriminationvector and an actual (real or fake) label of the design data is reduced.

That is, in a learning session of the UX-bit attribute discriminator152, the learning session may have a configuration in which parametersof the UX-bit attribute discriminator 152 are updated such that a UX-bitattribute discrimination vector D_(ux)(x,u) in a distribution of anencoding vector of design data in which an encoding vector of UX-bitattribute data is set as a reference becomes minimum (0), and a UX-bitattribute discrimination vector D_(ux)(u,u) in a distribution of anencoding vector of UX-bit attribute data, which is a ground truthdistribution corresponding thereto, becomes maximum (1). For example, aloss included in a loss function of the UX-bit attribute discriminator152 may be as follows.

L _(ux) =E _(x˜L) _(g) _(,u˜L) _(G) [log(1−D _(ux)(x,u)]+E _(u˜L) _(G)[log(D _(ux)(u,u)]  [Equation 13]

In Equation above, L_(ux) denotes a loss of the UX-bit attributediscriminator, x denotes an encoding vector of design data, u denotes anencoding vector of UX-bit attribute data which is reference data as aground truth, u˜L_(G) denotes a distribution of the reference data,x˜L_(g) denotes the distribution of the encoding vector of the designdata, and D_(ux)(x,u) denotes a UX-bit attribute discrimination vector(probability value between 0 and 1) output from the UX-bit attributediscriminator 152 when the encoding vector of the design data is inputusing the encoding vector of the UX-bit attribute data as a reference.D_(ux)(u,u) denotes a UX-bit attribute discrimination vector(probability value between 0 and 1) output from the UX-bit attributediscriminator 152 when the encoding vector of the UX-bit attribute datais input using the encoding vector of the UX-bit attribute data as areference. In a learning session of the UX-bit attribute discriminator152, when it is determined that the encoding vector of the design dataoutput by the design generation module 15 is not similar to the encodingvector of the UX-bit attribute data, the loss of the UX-bit attributediscriminator 152 is output such that D_(ux)(x,u) is close to 0, andL_(ux) is close to 0 (minimum value), and when it is determined that theencoding vector of the design data is similar to the encoding vector ofthe UX-bit attribute data, the loss of the UX-bit attributediscriminator 152 is output such that D_(ux)(x,u) is close to 1, andL_(ux) is close to ∞ (maximum value), which may be applied to learning(parameter update) of the UX-bit attribute discriminator 152. Inaddition, parameters of the UX-bit attribute discriminator 152 may beupdated in a direction in which D_(ux)(u,u) approaches 1. Regarding alearning session of the UX-bit attribute discriminator 152, the learningsession of the UX-bit attribute discriminator 152 may have aconfiguration in which learning data labeled as synthesized in thedesign generation module 15 and learning data labeled as not synthesizedare provided as leaning data of the UX-bit attribute discriminator 152,the learning data is input to the UX-bit attribute discriminator 152 tooutput a UX-bit attribute discrimination vector, a loss of the learningdata is calculated based on the UX-bit attribute discrimination vector,and parameters of the UX-bit attribute discriminator 152 are updated ina direction in which a calculated loss of the UX-bit attributediscriminator 152 is minimized. Alternatively, regarding a modifiedexample of a loss of the UX-bit attribute discriminator 152, a lossincluded in a loss function of the UX-bit attribute discriminator 152may also include a mean square loss, a cross entropy loss, and the like.

Regarding an operational relationship of the UX-bit attributediscriminator 152 in a learning session of the design generation module15, a loss function of the design generation module 15 may include aUX-bit attribute discrimination vector output when an encoding vector ofUX-bit attribute data is input to the UX-bit attribute discriminator 152as input data. In the loss function of the design generation module 15,for example, the UX-bit attribute discrimination vector may be providedas a loss (hereinafter referred to as UX-bit attribute discriminationloss) as follows.

L _(dux) =E _(x˜L) _(g) _(,u˜L) _(G) [log(D _(ux)(x,u)]  [Equation 14]

In Equation above, L_(dux) denotes a UX-bit attribute discriminationloss, x denotes an encoding vector of design data, x˜L_(g) denotes adistribution of the encoding vector of the design data, u denotes anencoding vector of UX-bit attribute data, z˜L_(G) denotes a distributionof the encoding vector of the UX-bit attribute data, and D_(ux)(x,u)denotes a UX-bit attribute discrimination vector (probability valuebetween 0 and 1) output from the UX-bit attribute discriminator 152 whenthe encoding vector of the design data is input as input data using theencoding vector of the UX-bit attribute data as a reference. In alearning session of the design generation module 15 performed by theUX-bit attribute discriminator 152, the learning session may have aconfiguration in which the UX-bit attribute determination loss includingthe UX-bit attribute discrimination vector is included in a lossfunction of the design generation module 15, and parameters of thedesign generation module 15 are updated such that the UX-bit attributediscrimination vector D_(ux)(x,u) in the distribution of the encodingvector of the design data becomes maximum (1) and thus L_(dux) becomes 0(minimum value) (such that it is determined that the encoding vector ofthe design data is similar to the encoding vector of the UX-bitattribute data).

Regarding another embodiment of the design generation module 15, FIG. 31is a schematic diagram illustrating an operational relationship of thedesign generation module 15 according to another embodiment of thepresent disclosure. As shown in FIG. 31 , the design generation module15 according to another embodiment of the present disclosure may beconnected to a nonlinear network. The linear network is a network havinga nonlinear structure in which a plurality of fully connected (FC)layers are consecutively connected. The design generation module 15 is amodule which receives a concatenated vector, which is obtained byconcatenating an image theme encoding vector and a text theme encodingvector, as input data, outputs a theme vector as output data, and inputsthe output theme vector to a plurality of layers in a network of thedesign generation module 15 for each scale. In this case, rather than aconcatenated vector obtained by concatenating the image theme encodingvector, the text theme encoding vector, and UX-bit attribute data,UX-bit attribute data is applied as the input data of the designgeneration module 15, and a noise vector is input to each layer having ascale to which a theme vector is input. Accordingly, since a datadistribution of the image theme encoding vector and the text themeencoding vector does not limit a data distribution of the designgeneration module 15, and concurrently, a data distribution of theUX-bit attribute data limits the data distribution of the designgeneration module 15, it is strict on the UX-bit attribute data, and adegree of freedom is increased from an image theme and a text themegiven by a practitioner, thereby obtaining an effect in which variousdesigns can be generated in a fixed wireframe.

Regarding a skip connection of the design generation module 15, FIG. 32is a schematic diagram illustrating the skip structure of the designgeneration module 15 according to one embodiment of the presentdisclosure. As shown in FIG. 32 , the skip connection of the designgeneration module 15 refers to a structure in which the image themeencoding module 10 and the design generation module 15 are connectedthrough an image theme encoding vector, and also a specific layer of theimage theme encoding module 10 and a specific layer of the designgeneration module 15 are additionally connected through addition logic.Accordingly, there is an effect in which gradient vanishing andexploding problems are reduced, and since only the residual of how muchchange from a previous layer needs to be calculated, there is an effectin which computing resources are reduced.

Regarding another embodiment of the theme encoding module 10, FIG. 33 isa schematic diagram illustrating a common theme segment module 101according to another embodiment of the present disclosure. As shown inFIG. 33 , the image theme encoding module 10 may be provided to beconnected to the common theme segment module 101. The common themesegment module 101 may be a module which receives image theme data 100as input data and common theme segment data as output data and may beprovided such that parameters of the common theme segment module 101 areupdated in a direction in which similarity between a text theme encodingvector and an image theme encoding vector is increased. As a networkstructure of the common theme segment module 101 according to oneembodiment of the present disclosure, a structure of a FullyConvolutional Network (FCN) by J. Long et al. (2015), a ParseNet by W.Liu et al. (2015), a Convolutional and Deconvolutional Networks by H.Noh et al. (2015), or a U-Net by Ronneberger et al. (2015) may beapplied. Accordingly, not all of image theme data input by apractitioner is used to generate design data, but only a part of theimage theme data related to text theme data is segmented and used togenerate design data, thereby obtaining an effect in which the designgeneration module 15 generates a design that is more relevant to anintention of an operator.

Through an organic combination of a configuration of an automatic designgenerating artificial neural network device 1 and a configuration ofUX-bits according to one embodiment of the present disclosure asdescribed above, there are effects in which UX-bit attributes can bedefined by a practitioner simply inputting image theme data which is animage corresponding to a concept and a theme of a design to be generatedand simply inputting text theme data which is text corresponding to theconcept and theme of the design to be generated, and an IA, a wireframe,and a web/app design can be generated.

As described above, those skilled in the art to which the presentdisclosure pertains will understand that the present disclosure may beimplemented in other detailed forms without changing the technicalspirit or indispensable characteristics of the present disclosure.Accordingly, it will be understood that the above-described embodimentsare illustrative and not limitative from all aspects. The scope of thepresent disclosure is defined by the appended claims rather than thedetailed description, and the present disclosure should be construed ascovering all modifications or variations derived from the meaning andscope of the appended claims and their equivalents.

The features and advantages described in the specification are not allinclusive, and particularly, many additional features and advantageswill be apparent to one of ordinary skill in the art in view of thedrawings, specification, and claims hereof. Moreover, it should be notedthat the language used in the specification has been principallyselected for readability and instructional purposes and may not havebeen selected to delineate or limit the subject matter of the invention.

The above description of embodiments of the present disclosure has beenpresented for purposes of illustration It is not intended to limit thepresent disclosure to the disclosed precise form or to describe thepresent disclosure without omission. Those skilled in the art canappreciate that many modifications and variations are possible inconsideration of the above disclosure.

Therefore, the scope of the present disclosure is not limited by thedetailed description and is limited by any claims of the applicationbased thereon. Accordingly, the disclosure of embodiments of the presentdisclosure is illustrative and does not limit the scope of the presentdisclosure set forth in the claims below.

1. An automatic design generating artificial neural network device usinga user experience (UX)-bit, comprising: an image theme encoding modulethat is an encoding module which receives image theme data, which is animage representing a theme of a web/app graphic design to be generatedby a practitioner, as input data, and outputs an image theme encodingvector as output data; a text theme encoding module that is an encodingmodule which receives text theme data, which is text representing thetheme of the web/app graphic design to be generated by the practitioner,as input data, and outputs a text theme encoding vector as output data;a UX-bit generation module that is a module which receives the imagetheme encoding vector and the text theme encoding vector as input dataand outputs UX-bit attributes of a plurality of UX elements as outputdata; a design generation module that is an upsampling artificial neuralnetwork module which receives the image theme encoding vector, the texttheme encoding vector, and the UX-bit attribute as input data andoutputs design data indicating a specific web/app graphic design asoutput data; an image theme discriminator that is a module used in alearning session of the design generation module and is a pre-learnedartificial neural network module which, when the design data and theimage theme data are input as input data, outputs an image themediscrimination vector, which indicates a probability of similaritybetween the design data and the image theme data, as output data; a texttheme discriminator that is a module used in the learning session of thedesign generation module and is a pre-learned artificial neural networkmodule which, when a design encoding vector that is an encoding vectorof the design data and the text theme encoding vector are input as inputdata, outputs a text theme discrimination vector, which indicates aprobability of similarity between the design encoding vector and thetext theme encoding vector, as output data; and a UX-bit attributediscriminator that is a module used in the learning session of thedesign generation module and is a pre-learned artificial neural networkmodule which, when the design encoding vector and an encoding vector ofthe UX-bit attribute are input as input data, outputs a UX-bit attributediscrimination vector, which indicates a probability of similaritybetween the design encoding vector and the encoding vector of the UX-bitattribute, as output data, wherein, in the learning session of thedesign generation module, parameters of the design generation module areupdated in a direction in which a representative design loss, which iscomposed of a difference between the design data and web/app designreference data (ground truth) in which similarity with the image themeencoding vector and similarity with the text theme encoding vector aregreater than or equal to a specific level in an encoding vector of apre-stored web/app design corresponding thereto, an image themediscrimination loss comprising the image theme discrimination vector, atext theme discrimination loss comprising the text theme discriminationvector, and a UX-bit attribute determination loss comprising the UX-bitattribute discrimination are reduced.
 2. The device of claim 1, furthercomprising a nonlinear network which is connected to the designgeneration module, and is a network having a nonlinear structure inwhich a plurality of fully connected (FC) layers are consecutivelyconnected, wherein the nonlinear network receives a concatenated vectorobtained by concatenating the image theme encoding vector and the texttheme encoding vector as input data, and outputs a theme vector asoutput data, and wherein the nonlinear network is module which inputsthe output theme vector to a plurality of layers in a network of thedesign generation module for each scale, wherein, in the designgeneration module, the UX-bit attribute is input as input data, and anoise vector is input to each layer having a scale to which the themevector is input.
 3. The device of claim 1, further comprising a commontheme segment module which is connected to the image theme encodingmodule and is a module which receives the image theme data as input dataand outputs common theme segment data as output data, wherein, in alearning session of the common theme segment module, parameters of thecommon theme segment module are updated in a direction in whichsimilarity between the text theme encoding vector and the image themeencoding vector are increased.
 4. The device of claim 1, wherein: theUX-bit attributes comprise UX-bit function attributes and UX-bit designattributes; and the UX-bit generation module comprises a UX elementgeneration module that is a module which generates the plurality of theUX elements to match the UX-bit function attribute and the UX-bit designattribute with a specific UX element, and an recurrent neural network(RNN) module that is an artificial neural network module which outputsthe UX-bit function attribute and the UX-bit design attribute for the UXelement.
 5. The device of claim 1, wherein: the UX-bit attributescomprise UX-bit function attributes and UX-bit design attributes; andthe UX-bit generation module comprises a UX element generation modulethat is a module which generates the plurality of the UX elements tomatch the UX-bit function attribute and the UX-bit design attribute witha specific UX element, and an RNN module that is an artificial neuralnetwork module which outputs the UX-bit function attribute and theUX-bit design attribute for the UX element, wherein: the RNN modulecomprises RNN blocks having a first RNN cell and a second RNN cell as abasic unit and the image theme encoding vector and the text themeencoding vector are used as initial input data; the first RNN cellreceives the initial input data or output data of a previous cell andRNN hidden layer information and outputs the UX-bit function attributeof an n^(th) UX element; and the second RNN cell receives the UX-bitfunction attribute, which is the output data of the previous cell, andthe RNN hidden layer information, and outputs the UX-bit designattribute for the UX-bit function attribute of the n^(th) UX elementoutput from the first RNN cell.
 6. The device of claim 1, wherein: theUX-bit attributes comprise UX-bit function attributes and UX-bit designattributes; and the UX-bit generation module comprises: a UX elementgeneration module that is a module which generates the plurality of theUX elements to match the UX-bit function attribute and the UX-bit designattribute with a specific UX element; an RNN module that is anartificial neural network module which outputs the UX-bit functionattribute and the UX-bit design attribute for the UX element; and areinforcement learning module which is configured such that the UX-bitattributes of all pre-generated UX elements, the image theme encodingvector, and the text theme encoding vector are input as an environment,an RNN block of the RNN module is set as an agent, a situation, in whichan n^(th) UX element having the UX-bit function attributes and theUX-bit design attributes is virtually included in the UX-bit functionattributes and the UX-bit design attributes of first to (n−1)^(th)elements, is set as a state, wherein, in such a state, the UX-bitfunction attributes and the UX-bit design attributes output for then^(th) UX element by the RNN block, which is the agent, are input for anaction, and as similarity is high between comparison information and theUX-bit function attributes and the UX-bit design attributes of then^(th) UX element which are output data, a relatively high reward isgenerated to update a hidden layer of the RNN block which is the agent,wherein the comparison information indicates a concatenation between theimage theme encoding vector and the text theme encoding vector.
 7. Anautomatic design generating method, performed by the device according toclaim 1, the method comprising: an image theme encoding operationperformed by an image theme encoding module that is a component of theautomatic design generating artificial neural network device, the imagetheme encoding module receiving image theme data, which is an imagerepresenting a theme of a web/app graphic design to be generated by apractitioner, as input data, and outputting an image theme encodingvector as output data; a text theme encoding operation performed by atext theme encoding module that is a component of the automatic designgenerating artificial neural network device, the text theme encodingmodule receiving text theme data, which is text representing the themeof the web/app graphic design to be generated by the practitioner, asinput data, and outputting a text theme encoding vector as output data;a UX-bit generating operation performed by a UX-bit generation modulethat is a component of the automatic design generating artificial neuralnetwork device, the UX-bit generation module receiving the image themeencoding vector and the text theme encoding vector as input data andoutputting UX-bit attributes of a plurality of UX elements as outputdata; and a design generating operation performed by a design generationmodule that is a component of the automatic design generating artificialneural network device, the design generation module receiving the imagetheme encoding vector, the text theme encoding vector, and the UX-bitattribute as input data and outputting design data indicating a specificweb/app graphic design as output data, wherein: a learning session ofthe design generation module comprises: an image theme discriminatingoperation performed by an image theme discriminator that is a componentof the automatic design generating artificial neural network device,when the design data and the image theme data are input as input data,the image theme discriminator outputs an image theme discriminationvector, which indicates a probability of similarity between the designdata and the image theme data, as output data, a text themediscriminating operation performed by a text theme discriminator that isa component of the automatic design generating artificial neural networkdevice, when a design encoding vector that is an encoding vector of thedesign data and the text theme encoding vector are input as input data,the text theme discriminator outputs a text theme discrimination vector,which indicates a probability of similarity between the design encodingvector and the text theme encoding vector, as output data, and a UX-bitattribute discriminating operation performed by a UX-bit attributediscriminator that is a component of the automatic design generatingartificial neural network device, when the design encoding vector and anencoding vector of the UX-bit attribute are input as input data, theUX-bit attribute discriminator outputs a UX-bit attribute discriminationvector, which indicates a probability of similarity between the designencoding vector and the encoding vector of the UX-bit attribute, asoutput data; and in the learning session of the design generationmodule, parameters of the design generation module are updated in adirection in which a representative design loss, which is composed of adifference between the design data and web/app design reference data(ground truth) in which similarity with the image theme encoding vectorand similarity with the text theme encoding vector are greater than orequal to a specific level in a pre-stored web/app design correspondingthereto, an image theme discrimination loss comprising the image themediscrimination vector, a text theme discrimination loss comprising thetext theme discrimination vector, and a UX-bit attribute determinationloss comprising the UX-bit attribute discrimination are reduced.
 8. Anautomatic design generating artificial neural network system comprising:a practitioner client which receives image theme data that is an imagerepresenting a theme of a web/app graphic design to be generated by apractitioner and text theme data that is text representing the theme tobe generated by the practitioner from the practitioner; and theautomatic design generating artificial neural network device using aUX-bit according to claim 1 which receives the image theme data and thetext theme data from the practitioner client and outputs design datacorresponding to the image theme data and the text theme data.